As citizens of a dangerous world, we all face security and safety risks. Every day, 30 people die by gunshot in the U.S.—one every 48 minutes. A police officer dies from a gunshot wound every ten days. Analysis of past video data may save lives.
A recently foiled terrorist attack on Ft. Dix Army Base in New Jersey involved five terrorists planning to kill U.S. soldiers at the army base. They were observed in video cameras surveying the army base on numerous occasions prior to the planned attack. A well-meaning citizen notified the police and FBI by submitting a “video tip” which started an investigation. The video tip was a video of the men training for the terrorist attack and plotting to kill as many American soldiers in as short a time as possible. Accordingly, the military is concerned about historical analysis of past video data, as well as data from video tips.
Muggings and home intrusions are another threat to citizens. In Seattle, Wash. one in every 60 homes was burglarized in 2006. In Boston, Mass. in 2007 an 87-year old woman opened her home only to find a burglar in her home. Proactive alerts based on past video data may deter such crimes.
Vandalism and damage to property decreases property values. One study conducted by the London School of Economics found that “a one-tenth standard deviation increase in the recorded density of incidents of criminal damage has a capitalized cost of just under 1% of property values, or £2,200 on the average Inner London property” (Steve Gibbons, The Costs of Urban Property Crime, 2003). Analysis of current and past video data may prevent such vandalism.
Violence in schools and on college campuses continues to rise, and has increased concern among students, parents, and teachers. A shooting at Virginia Tech University in 2007 resulted in the killing of 32 people and injured 24 others. In 2005, a professor at MIT was shot four times in a parking lot on campus. If the video data was stored and analyzed using meta-data, the assailants could have been apprehended. The shooting may have even been thwarted.
Serious accidents at corporate facilities have resulted in enormous damage to personal lives and to corporate property. For example, an explosion in a Texas oil refinery killed 15 people and injured 180 others. The U.S. Chemical Safety Board determined that various factors, one of which was the absence of adequate experience in the refinery, contributed to the accident: “As the unit was being heated, the Day Supervisor, an experienced ISOM operator, left the plant at 10:47 a.m. due to a family emergency. The second Day Supervisor was devoting most of his attention to the final stages of the ARU startup; he had very little ISOM experience and, therefore, did not get involved in the ISOM startup. No experienced supervisor or ISOM technical expert was assigned to the raffinate section startup after the Day Supervisor left, although BP's safety procedures required such oversight.” (Chemical Safety Board, Investigation Report: Refinery Explosion and Fire, March 2007, pg. 52.) Video surveillance, storage, and analysis could have prevented these deaths and injuries.
As a result of terrorist activity (such as the attempted terrorist attack on Ft. Dix), violence on college campuses (such as the shooting at Virginia Tech University), and major accidents (such as the oil refinery explosion in Texas), governments, corporations, universities, other institutions, and individuals are increasingly concerned about security and safety. To address this problem, many of these institutions are installing security and surveillance cameras around their facilities, campuses, and military installations.
Once the video data is captured by these cameras, which could be analog or digital cameras, the video data has to be stored, and subsequently retrieved, and information about the quality of the images also has to be stored. There are numerous problems with conventional video data storage and retrieval systems. For example, conventional video data from analogue cameras that is stored on VHS tape is difficult to store and retrieve. The VHS tape has to be rewound multiple times to search for a particular occurrence. This can damage the VHS tape, by stretching the VHS tape and scraping the polymer coating.
Digital video data from digital cameras may be stored in digital, random-access media, such as disk. Unfortunately, the vast amount of data generated by digital video cameras is also difficult to store, search, and retrieve from disk. For example, a typical 3 Megapixel digital surveillance camera generates images of approximately 280 Kbytes per frame. If this camera were running at 5 frames per second, it would generate approximately 60 GB per day. If an organization wanted to archive the data for one month, it would take approximately 1.8 TB, and if the organization wanted to archive the data for one year, it would take approximately 22 TB. In a typical application having 100 surveillance cameras around a particular facility, this translates into approximately 6 TB per day, or approximately 180 TB per month, or over approximately 2,000 TB per year! This is a large amount of data to store, search, and retrieve by traditional mechanisms. Present systems cannot store, archive, search, and retrieve such large amounts of data effectively and intelligently. When a pro-active alert that depends on past video data needs to be issued to deter a crime or other dangerous event, or past video data needs to be forensically analyzed for a past crime or other dangerous event, the inadequacies of present systems is even more apparent.
One drawback with conventional video storage is that the video data is only indexed by date and time. Therefore, an operator must know the date and time of events of interest before being able to search for those events.
Once the video data has been stored, another drawback with conventional video storage is the inability to perform intelligent search. For example, present systems cannot perform search by various meta-data criteria, such as “show all times when 2 or more people were detected in a given area.” Another drawback with conventional video storage is the inability to perform a search that retrieves video data across multiple locations and cameras. For example, present systems cannot perform a search such as “show all times when there was a gunshot detected at this location, and 2 or more people were detected in an adjacent area.”
Another drawback with conventional video storage is that all video data is weighted equally. For example, motion detected in an ammunition storage area of an army base would be weighted equally to motion detected in the basement of a dinning hall of the army base. In addition, video data from an old, low quality camera would receive the same weight as video data from a new, high quality camera.
Once the video data is stored, another drawback with conventional video storage is data security and integrity. Anyone who has physical access to the disk or tape can damage it, destroying potentially valuable evidence. For example, after a shooting on MIT's campus, the District Attorney's office gained access to the surveillance tape, deleted the video of the shooting, deleted date and time stamps from the tape, and rearranged the remaining images to portray a different set of actions, as well as permanently damaging the original tape. Another drawback with conventional video storage is the difficulties associated with archiving the video data.
Another drawback with conventional video storage is the inability to audit the video data, for example, determine who viewed the video data, and thus provide for audit of the video data.
Another drawback with convention video storage and analysis is the inability to utilize tips. Tips, that is, information from informants, are an important source of data. With the proliferation of video phones (cell phones with integrated cameras), tips are increasingly received as video clips captured at the scene of a crime by well-meaning citizens.
These drawbacks can be overcome with the attendant features and advantages of the present invention. Therefore, as recognized by the present inventors, what are needed are a method, apparatus, and system for storing, searching, archiving, protecting, auditing, and retrieving video data and associated meta-data and attribute data.
What is also needed is a method for monitoring and auditing the stored video data as well as live video data. What is also needed is a method for intelligent alerting of appropriate individuals based on stored video data as well as the live video data.
Accordingly, it would be an advancement in the state of the art to provide an apparatus, system, and method for storing, searching, auditing, and retrieving video data received from multiple cameras, and for generating intelligent alerts based on the stored video data.
It is against this background that various embodiments of the present invention were developed.